{
 "cells": [
  {
   "cell_type": "raw",
   "id": "f82f07cc",
   "metadata": {},
   "source": [
    "对于定价示例，让我们考虑一个工作负载，其中引入速率为 15 MB/s，复制因子为 3，并且您希望在 Kafka 集群中保留数据 7 天。\n",
    "\n",
    "对于这样的工作负载，它需要 6 个 m5.large 代理，具有 32.4 TB EBS 存储，成本为 4，755 美元。\n",
    "\n",
    "但是，如果您将分层存储用于相同的工作负载，本地保留 4 小时，总体数据保留 7 天，则需要 3 个 m5.large 代理，具有 0.8 TB EBS 存储和 9 TB 分层存储，成本为 1，584 USD。\n",
    "\n",
    "如果要一次读取所有历史数据，则费用为13美元（每GB检索成本为0.0015美元）。\n",
    "\n",
    "在此分层存储示例中，您可以节省大约 66% 的总体成本。"
   ]
  },
  {
   "cell_type": "raw",
   "id": "2f87cd4e",
   "metadata": {},
   "source": [
    "https://aws.amazon.com/cn/msk/pricing/\n",
    "\n",
    "美国东部区域\n",
    "\n",
    "kafka.m5.large   0.21 USD\n",
    "\n",
    "0.10 USD（美国东部区域每月每 GB 价格）\n",
    "\n",
    "*此工作负载反映摄取速率为 100 KB/秒 且保留 24 小时、复制因子为 2 时的情况。将产生数据传输费用，预计此工作负载的费用为每月 5 USD。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "5af0910f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ec2 cost: 937.44\n",
      "storage cost: 3317.76\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "4255.200000000001"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kafka_cost=0.21*24*6*31\n",
    "print('ec2 cost: {}'.format(kafka_cost))\n",
    "storage_cost=0.1*32.4*1024\n",
    "print('storage cost: {}'.format(storage_cost))\n",
    "kafka_cost+storage_cost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "3825cd15",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "26578.125"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "15*1024/100*5\n",
    "\n",
    "15*3*86400/1024*7"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8c12678",
   "metadata": {},
   "outputs": [],
   "source": [
    "--conf spark.sql.adaptive.enabled=true \\\n",
    "--conf spark.sql.adaptive.shuffle.targetPostShuffleInputSize=209715200 \\\n",
    "--conf spark.sql.adaptive.advisoryPartitionSizeInBytes=209715200 \\\n",
    "--conf spark.sql.adaptive.coalescePartitions.minPartitionNum=1 \\"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "python",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
